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ABSTRACT 

The need for an eapirically defensible aeans of 
sequencing instruction appears to have been the priaary aotivator for 
research into learning hierarchies. Also, valid learning hierarchies 
could act as congealing forces in individualizing instructional 
systeas by providing psychological roadaaps for diagnosing students' 
preinstructional skills and aonitoring students' progress through 
instructional systeas. Four aethods for generating candidates for 
learning hierarchies are available: introspect! on » foraal analysis* 
observation, and statistical "fishing." Experimental transfer of 
training studies and statistical studies have been used to evaluate 
posited hierarchies. White recently aade significant aod'ifications of 
these standard evaluation aethods to iaprove the internal validity of 
research into learning hierarchies. Several external validity issues 
reaain to be revived before ^arce resources should continue to be 
allocated for additional learning hierarchy research. (Author) 
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VALIDATION OF LEARNING HIERARCHIES 
FOR OBJECTIVE-BASED INSTRIKITKMIAL SYSTEMS 



David Lynn Passmore 
University of Massachusetts /A^erst 

Focus of the Paper 

Gagne (1968) considered the selection of a sequence for presenting 
learning events to be the most iiiq>ortant decision in the design of an 
instructional system. This paper describes the nature of instructional 
sequencing research coai>leted in the last 10 to 12 years that has 
been based on Gagne 's studies of learning hierarchies. The {.urposes 

of this paper are to: 

(1) synthesize the major reasons for interest in learning 
hierarchy research; 

and (2) briefly describe methodologies available for this type 
or research. 

this paper is organized under three major topics. First, factors 
in American Intellectual heritage and in the development of instruc- 
tional systems theory are related to the emergence of interest in 
learning hierarchy research. Second, methods for generating and 
evaluating learning hierarchies are reviewed. Finally, the previous 
two headings are summarized and some prescriptive statements are made 
concerning the future of learning hierarchy research. 

Reasons for Interest 
in Learning Hierarchies 

roTty years before he became president, John Adams was a young 



sdiooliaaster in Worcester, Massachusetts* He kept a now-famous diary 
in %fhich he tried to synthesize his ideas and experiences. Everything 
vent into his record — the weather, teas, and dinners, conversations, 
his thou^ts on sermons he heard and books he refi^-H>ut the welter 
of detail organized itself into an unplanned, bu; discernible, rhytha. 

During these apprmitice years in Worcester, Mam enacted a quiet 
and private drama. Me was recently graduated froa Harvard, not yet 
ed}arked on a career, and vulnerable to alternating 8»i^nts of des- 
pondency aad hope. But, ev^ in the privacy of his c\' he trans- 
lated his prc^leos into very public and traditional Cii^. dories. 
The diary became a sort of morality play in which Mism ' elations and 
fears paraded behind msks of abstract ideas about human nature and 
the world at large. 3n <me side was the ancient notion of man's 
erratic frailty. On the other was a confidcmce, almost as ancient, 
in the fundamental rationality of God's creation. For Adams, and 
other Amerlcant^ confidence in the orderlines of nature provided 
assurance of the continual and vigilant providence of God. 

Belief in the orderliness of nature has been a kernel idea in 
our intellectual history from Greek tia^s to Adam's to the present. 
It has been an organizing concept in the sciences. This belief is 
ev'ldent in l^xwin's On the. OfUgAM tkz Spe.CA.&i, in Malthus' E66ay 
an VfUncLptt Pojpul&tLon, and in other i^»ortant essays. It 
has been deeply rooted in the core of American tradition in the 
physical and natural sciences as well as economics. Education was 
also influenced by this thinking. If nature operates in an orderly 
manner f then the most effective methods of conducting the teaching/ 



learning enterprise could be uncovered using the scientist's tools. 
The study of edutMticn as a science is founded on these principles. 

Horace Mann and other early nineteenth century school refonwsrs 
evangelically strove to channel generalized faith in education into 
support for the coiwion school. The next generation of schoolmen, 
more bureaucrats than evangelists, believed that regimentation and 
standardization were necessary to unify support for public education. 
This belief led to the search for the "one best way", a fuUional 
system, for organizing structiai. The zeal which characterized 
this search is evident in Philbrick's (1885) hyperbole: "If America 
devised the best school desk, it must go to the ends of the civilized 
world [p. 58]." 

At the beginning of the twentieth century, reform-conscious 
America discovered "scientific management' with the help of Frederick 
W. Taylor (1911). "Scientific management" principles Injected the 
concept of efficiency into American business practices by developing 
methods for maximizing industrial outputs for a minimum of Inputs 
and. thereby, minimizing the cost of the enterprise and maximizlnR 
the return on capital invested. Callahan (1962. esp.. p. 23 but also 
paA6lm) documented the spread of applications of "scientific manage- 
nffint" dogma to household tasks, family duties, church functions, and, 
ultimately, the process of education. Not only were the iiK>8t effec- 
tive instructional methods pursued, but the dla»nslon of cost- conscious- 
ness was also added. The study of the economics of education is 
founded on these principles. 

Armed with tools such as systems theory, educational tests, and 
cost/effectiveness analysis, instructional systems experts entered 



the educational research arena in the liturgical spirit of the educa- 
tional efficiency movettent. (kie research problen attacked by th^se 
research workers was the deternination of the most effective and 
efficient ways to sequence learning events. 

Vt\?elopmnt oi Tf^ViuctLonoLl Sy6t^ The^oAy 

One nacro-level isst^ in curriculun design is the deternination 
of the proper sequence aaong elements of a set of desirable educa- 
tional experiences. For esanplei should students receive instruction 
in American History before they study civics? ^ould instruction in 
English grammar precede the study of EngUsh literature? Should 
students learn to weld in school-based occupational programs before 
they engage in on-the-job welding training programs? These broad 
sequencing questions, as well as antecedent questions concerning 
curricular scope and content, may find imswers tiirou^ professional 
analysis and pti>lic debate of the aims and means of schooling. Unique, 
cons^sus solutions to the problens posed by these and other general 
curriculinn questions have not ^aerged although gallons of academic 
blood have been shed by curriculum theorists in argus^ts over sub- 
stitutable problem solutions. 

Rather than deal with molar curricular issues, instructional 
systems technicians have purst^d answers to i^re micro-level problems. 
One popular toiKihstone of quality in current instructional systems 
theory is the requirei^nt that desirable learning outcomes for students 
exiting an instructional system be stated in precise, intersubjectively 
observ^le terns (Lawson, 1974). Instructional systems using rhis 
approadi are often called objective-based Instructional systems. 

Designers of objective-based instructional systems are confronted 
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%rf.th a lArge nui^er of technical problems, oie of *^ich is selecting 
the tei^ral order of presentation of learning events. For exaople, 
must Instruction In addition of single digit nunbers precede lessons 
In the addition of nuabers with nultlple digits? This sequencing 
problem Is analogous to the currlculuBi '^eorists* sequencing dlleoBa. 
Naturally, the two iptableaa differ In i.e. Also, Instructional 
systenis designers* sequencing prc&l^as should be less elusiw, less 
value-laden, and store answerable by direct empirical evidence than 

are sltailar sequotcing probleas faced by curriculum workers. 

Standard solutions to the problem of sequencing learning events 
in objective-based instructional systems have been provided for 
various school subject matter areas by: (1) identifying important 
behaviors that must be mastered for competence in a particular sub- 
ject matter area; <2) stating intended learning outcomes, or educa- 
tional objectives, which precisely state the behaviors which mist 
be a»8tered as well as the degree to trtiich and conditions under which 
the b^aviors are to be exhibited by students; and (3) determining 
whether students* mastery of one objective depends upon their aastery 
of other objectives. Gagne (1970, Ch. 4) assigned the term **leaming 
hierarchy*' to designate the set of depoidencles a^mg co^onent skills 
within a learning ti^k. In objective-based instructional systems, 
precisely stated educational <^Jectlves define the coaponeat skills 
of learning tasks. An important assumption is that it is thotight 
to be mora appropriate to organize instruction by using functitmal 
dependencies among objectives in a learning hierarchy than it is to 
use logical, intuitive sequencing strategies (Cf. however, Brlggs, 
1967 for other sequencing viewpoints). 

o 
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LearniPg hierarchies discovered throu^ this three-part analysis 
serve at leaat three functloas In object lve-4»a8ed instructional aystess. 
First, from a design angle, they ml^t suggest the aost efficient 
ways to sequence learning events. Second, a diagnosis of a student's 
nastery of the coaponent skills In a learning hierarchy aay guide 
decisions concerning the sttKlent's entry intt^ the ins truest lonal se- 
quence. This acknowledges that students differ with respect to their 
mastery of the conponent skills in the learning hierarchy as they 
enter the Instructional systea and provides a nechanlsm for individ- 
ually-tailoring Instruction to the student's pre-instructional skills. 
Third, learning hierarchies are a tool used in criterion-referenced 
assessaent plans for aonltorlng stiKlent progress througjb objective- 
based Instrucclonal systens. So, as Gagne (1968) asserted, a valid 
learning hierarchy could be a most useful device for individualizing 
an objective-based Instructional system* 

It follows that specialists in Instructional systems as well as 
In educational measurement havf? shown particular interest in learning 
hierarchy research. Instructinal systems specialists' interest in 
Instrtictlonal design issues motivates their study of learning hier- 
archies. Robert Gagne and his associates performed pioneering 
research into the development and validation of learning hierarchies 
(gagne & Bassler, 1963; Gagne, Mayor, Garstens, & Paradise, 1962; 
Gagne & Paradise, 1961; Gagne & Staff, 1965). Gagne's studies seem 
to have crystallized the methodological directions for subsequent 
efforts in learning hierarchy research by instructional systems 
specialists (White, 1973, p. 367). 



Some educational ateasureoent specialists are interested in the 
unique contributions that criterion-refer«iced assessacnt theory could 
make in the diagnosis of. and subsequent instructional placeaent on 
Che basis of. relevant pre- instructional skills. Some of these spe- 
cialists also believe that any strong criterion-referenced assessaenc 
theory should be applicable to making decisions concerning learners* 
progress throu^ instructional sequences (Llndvall & Cox, 1969) . 

Most of the work coaq>leted on criterion-referenced assessment 
theory has had. at best, a pre-scientiflc character (e.g., C:x. 1970; 
Cox & Vargas. 1966; Popham & Husek. 1969). However, more coherent 
specifications of measurement theory necessary for criterion-refer- 
enced assessment were presented by Hambleton end Novick (1973) and 
Swaminathan. Hantoleton. and Algina (in press), ine^decision-theor- 
etic Vase of the Hai^leton. U at, approach articulates well with 
the typical needs of existing instructional system monitoring oper- 
ations reviewed by Haafcleton (in press) . 

Among the novel approaches to monitoring student progress through 
the use of criterion-referenced tests in objective-based instructional 
programs is the technique of branched, sequential, or tailored testing 
(Lord. 1968; Ferguson. 1969; Wood. 1973). Tailored testing has been 
defined as a strategy for testing in which the sequence of test items 
(Splnetl & Gambleton, 1973, p. 2). 

In suaanary. it appears that the need for an OTpirlcally defensi- 
ble means of sequencing learning events in Instructional systems has 
been the primary motivator for research into learning hierarchies. 
Also, valid learning hierarchies could act as congealing forces in 
individualising instructional systems by providing psychological 



roadmaps for diagnosing students* pre-instructlonal skills and for 
«onitoring students* progress through instructional systests. 

Methodologies Available for 
Generating and Evalttatlng Learning Hierarchies 

A three-part process for conducting sequencing research in the 

design of objective-based instructional systaas was outlined in Che 

previous section. In the present section, attention is focused on 

the third part of this process which is the deterainatlon of whether 

students' nastery of one objective depends upon their mastery of other 

objectives. Specifically, TOthods for generating candidates for 

learning hierarchies and for evaluating these posltl^ hierarchies 

are reviewed. Similar methodological reviews may be found in Briggs 

(1967), Resnick (1971), Resnlck and Wang (1969), Smith (1972), 

Ualbesser and Eisenberg (1972). and White (1973, 1974b). 

Hie first two parts of the instructional sequencing research 
process are treated lightly In this section. Methods for identifying 
Important behaviors that nntst be aastered for coa^tsnce in a parti- 
cular subject matter area and for expllclty stating Intended learning 
outcomes are not ^irect»i reviewed even though they are laportant 
steps In InstriKtional sequencing research. 

Techniques for selecting Inportant behaviors to be learned in an 
instructional system are usually classified under the topic of task 
analysis. Some methods for task analysis also yield plausible candi- 
dates for learning hierarchies and will, therefore, be review^ in 
this seclon. Conventional wisdom in explicitly stating Intended 
learning outcomes is recorded in Aaaerman and Melchlng (1966), 



Gronlund (1970), and Mager (1962). It will be suggested that fixation 
on this conventional wisdon aay subvert the discovery of valid learning 



hierarchies. 



A provisiooal ordering of instructional objectives must be posited 
before learning hierarchy research can proceed for a particular instruc- 
tional system. How oay these candidates for learning hierarchies be 
generated? Each researcher uses soae n^thod to coaplete this task 
even though the range of available methods have not been well documented. 
And it could be disastrous to operate without a hunch about the starting 
point for this task. Consider that there are fe! possible Un^ 
orderings for fe objectives and that the nusfeer of cmicelvable hierarchies 
capable of being generated from fe objectives is wuch larger. The 
sources of hunches about plausible hierarchies nay be grouped into four 
categoriess (1) introspection; (2) formal analysis; (3) <*servatlon; 
and (4) statistical "fishing." 

IntAOApectcon. One popular a«tbod for generating candidates for 
learning hierarchies is to ask the question, ••«hat would an individual 
need to know or do to display competence In this subject matter?" This 
same question, paraphrased from Gagne and Paradise (1961, p.4), is 
applied again to the original answer and. then, successively to subse- 
quent answers. This method of questioning produces a learning hierarchy 
characterized by general, molar behaviors at the top of the hierarchy 
and more specific, subordinate, and molecular behaviors near the bottom 
of the hierarchy (e.g.. Gagne. Mayor. Garstens. & Paradise, 1962. p.4). 
This introspective method of hierarchy generation most frequently 



ar pears in the literature with the instructional systcn developer the 
taost prominent actor in this process. There are variations on this 
theme. For example, subject satter experts (Payton, 1971) and students 
(Kaplan, 1964) have been asked to play introspective roles to generate 
learning hierarchies. 

fomal (maly6l&. Fonoal analyses of subject natter domains pro- 
vide another source of learning hierarchies. Good examples are the 
industrial manufacturing and ccmstructlon domains developed by the 
Industrial Arts Curriculum Project (Towers, Lux, & Ray, 1966). Each 
of these subject matter domains was organized as a hierarchical taxo- 
nomy. Levels of each hierarchy proceed from general to more specific 
elements of manufacturiag and construction practice. Within each 
level, elements were devised to be individually inclusive and mutually 
exclusive. 

Hierarchies suppll*»d by the introspective method previously 
discussed are developed by imagining the learner's interactlcm with 
the si&ject matter. In contrast, the method of formal analysis removes 
consideration of the learner and, instead, applies the logical rigor 
of taxonomic inquiry that has been developed for the natural sciences 

(see Gregg, 1954). 

ObiVivation, Learning hierarchies may also be posited by observing 
the natural order in which learners acquire behaviors. For exaa«>le. 
a problem could be assigned to learners and, then, the «ilestones in 
their progress toward the problem solution could be recorded. Perhaps 
observational methods similar to those described by Plaget and Inhelder 
(1969) for researching hierarchies among developmental tasks might be 
fruitful in learning hierarchy research. 



-11- 



While the method of observation introduces a realistic glimpse 
at learners as a source of learning hierarchies, it must be remembered 
that the sequence which learners choose to solve u problem may prove 
to be neither effective nor efficient. And maximum effectiveness and 
efficiency in facilitating laaming are primary quality criteria for 

Instructional systems. 

Stati&tLcaZ "^iA/vcng." Also, candidates for learning hierarchies 
are often captured through statistical studies which may be best char- 
acterized as fishing expeditions. Kie prior absence of a hypothesized 
network of educational objectives is a distinguishing feature of these 
types of studies. The main aim of these statistical studies is to 
suggest ©ttpirically-based. po^t ho<L, structural hypotheses about the 
relationships among a set of behaviors. 

Many of the numerical techniques, such as hierarchical cluster 
analysis (Tryon. 1958; McQuitty. I960; Ward. 1961) and hierarchical 
factor analysis (Coombs & Satter. 1949; Palmer & IfcCormick. 1961). 
applied to military task analysis could be used to fish for provis- 
ional candidates for learning hierarchies. Baker (1972). Chenzoff 
(1964). Cotterman (1959). FoUey (1964). Morsh. Madden, and Christal 
(1961) reviewed some of the more standard numerical approaches to 
the development of hierarchical taxonomies of behavior. A conference 
report by Moss and Smith (1970. esp. papers by Ammerman. Christal, 
and Silverman) described the application of military task analysis 
techniques to the constructim of vocational education curricula. 
In addition, a rich and suggestive source of nua«rical taxonomy 
methods that could be applied in learning hierarchy research may be 
found in the natural science literature (For starters see, Sokal & 
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Sneath. 1963; Sneath & Sokal, 1962). Driver^ (1963) survey of 
numerical clasalfication methods in anthropology might also be use- 
ful. 

One prominent line of thought in learning hierarchy research 
is that functional dependencies among Instructional objectives can 
be discovered by examining the Guttman (1944) scalability of the test 
items measuring the behaviors specified by the objectives. Since 
tt« success of Guttman' 8 procedure demands a linear ordering of item 
which measure a unidimensional entity, some researchers have pre- 
ferred to use Lingoes (1963) multidimensional extension of Guttaan's 
procedure to uncover non-linear orderings among educational objectives. 
Airasian and Bart (1971) and Bart and Kruz (1973) developed a more 
parsimonious technique for discovering hierarchies among test items. 
Applications of the Bart at. ordering-theoretic method were pre- 
sented in Airasian and Bart (1974), Bart (1972a, 1972b) and Bart and 
Airasian (1972).^ Because it is relatively new and could have strong 
applications learning hierarchy research, the Bart a£. method is 
briefly reviewed. 



^A computer program which implements to ordering-theoretic method 
and is compatible with the CYBER 76 awdel computer is available from 
W.M. Bart, Psychological Foundations of Education, 330 Burton Hall, 
University of Minnesota, Minneapolis, Minnesota, 55455. 
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A universe of response patterns to two binary-scorable (0« 
answer incorrect; 1- answer correct) test itens is enumerated below: 

""l » (0.0); 
^2 " (1,1) J 
''S - (1.0); 
and ^4 « (0,1). 

Bart, U at., described vectors ^1, ^2. and ^3 as coniOmaXofus response 
patterns; performance on the first test item is considered prerequisite 
to performance on the second item. On the other hand, response vector 
^4 is said to (LUcoviiVvm the prerequisite relationship between items 
one and two. 

Suppose that two items were administered to N examin-es If the 
percent of the N examinees exhibiting a disconfinaatory - n pattern 
18 less than some tolerance percent, then item <me is said to be pre- 
requisite to item two. A low tolerance percent, often one to five 
percent, is recommended. 

The Bart. U al,, ordering theory may be generalized to I items. 
The observed percent of disconf irmatory response vectors for all two 
item pairs in a set of I iteiBS may be displayed in an I x I matrix 
indexed along the rows and colunms by the item nunbers. The prere- 
quisite relationships among any of the 1 items may be determined by 
examining the disconf irmatory response matrix and can be portrayed 
by means of a line graph joining the prerequisite Item pairs. The 
line graph is the candidate for further validation as a learning hier- 
archy. 
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EvcdmUng Posittd HlifuVictUu 

An ordering of objectives posited for an instructional system 
must be evaluated to determine whether it should be rejected, modified, 
or adopted. Experimental transfer of training studies as well as 
statistical studies have been used in this evaluation task. Both 
types of studies were extensively reviewed and heavily criticized by 
White (1973, 1974a, 1974b) who, as a result, presented (tAiite, 1974c; 
White & Clark, 1973) significant modifications of earlier paradigms 
for research into learning hierarchies. 

StandoKd &\Hiluatlon meXkod&. A transfer of training effect is 
the action that learning one task has upon std>sequent learning or 
performance of another task- (Andreas, 1972, p. 439). Networks of 
positive — that is, beneficial and facilitating — transfer among a 
set of instructional lAJectives are frequently sought in the evalua- 
tion of learning hierarchies for instructional system design. The 
definition of a hierarchical relationship as positive transfer among 
learning events was used in Gagne's pioneering studies which set the 
tone ror the learning hierarchy research efforts that followed. 

How mi^t mastery of one instructional objective facilitate the 
mastery of another objective? Several hypotheses are reasonable. 
First, some IdtvvUcal behaviors may be required to master both objec- 
tives. Work habits or specific subject matter skills are examples 
of identical behaviors that might precipitate transfer effects. Second, 

^A review of papers by Airasian and Bart (1974), Durrell (1974), 
Huntley (1974), Phillips (1974), Roudabush (1974), Smith (1974), and 
Taylor, Bart, and Rosehel (1974) could have contributed to the com- 
prehensiveness of this section but copies of these papers were unavil- 
id>le during the preparation of this paper. 

ERIC 
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posltive transfer may occur when UmiWi stlnuli and responses are 
involved in the mastery of both objectives. The psychological mech- 
anisms operating to induce transfer in this case would be sti^uxlus 
and response generalization. 

Transfer of training studies are usually conducted in controlled 
settings. In the application of such studies to learning hierarchy 
research, factors such as warm-up effects and transfer of general 
work habits must be partitioned fr««i the transfer of specific subject 
matter skills to unequivocally answer the question "What is learned?". 
Murdock's (1957) evaluative review of numerous designs used in transfer 
experiments might be helpful to learning hierarchy researchers as 
may the review and critique of methods for measuring transfer effects 
provided by Gagne, Foster, and Crowley (1948). 

In addition to experimental studies, the same methods described 
as methods for statistical "fishing" for learning hierarchy candidates 
are also applied in the evaluation of posited hierarchies. Rather 
than using these statistical techniques to hypothesize hierarchies, 
the focus, instead, is on testing hypothesized hierarchies. 

An interest in validating learning hierarchies not only implies 
that information will be obtained to discern the appropriateness of 
the posited hierarchies, but also that the hierarchies will be improved 
if necessary. However, the range of decisions and decision-making 
strategies for Improving learning hierarchies have not been carefully 
delineated. For example, one decision that could be made on the basis 
of experimental or statistical evaluative evidence is that the objec- 
tives need to be reordered. Another decision that could be made based 
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on the same data nig^t be that the objectives need to be subdivided, 
refined, and, then, reordered. But an explicit method for rationally 
choosing between these competing decisions is lacking but could be a 
fruitful line of future research. Methodologies indepaidently developed 
for the formative evaluation of educational products might suggest 
solutions to this problem (see Passmore, Asche, & 0*Kelley, 1974). 

Incidentally, Hively (1972), asserted from a radical behavioriat's 
viewpoint that traditional methods of stating instructional objectives 
(so-called "behavioral" objectives) usually lead to the specification 
of too many classes of behavior within an rf>jective with one result 
being muddled and untestable hierarchies among objecfives. Hively. 
Patterson, and Page's (1968) use of item forms was an attempt to 
clarify more homogeneous intended learning outcomes for instructional 
systems. So, following conventional wisdom in the statement of 
instructional objectivce may lead to problems in the evaluation of any 
posited hierarchy an»ng these objectives. 

Wkit&'6 m)dL^4.a<vUom. White (1973, p. 371; 1974, p.l) felt that 
most evaluations of laarning hierarchies were hindered by the following 
six problems: 

(1) small sample size; 

(2) imprecise specification of hierarchy elements; 

(3) the use of only one question per eles^nt to test the 
dependencies an^ng elen^nts; 

(4) the absence of a test of hierarchical Independence 
that takes error of s^asurement into account; 

(5) the practice of testing for dependencies among hierarchy 
elements a^tVi instruction is completed on all elements, 
thus confounding forgetting with a lack of connection 
between elements; 
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and (6) lack of face validity for hierarchies which have been empir- 
ically evaluated and ioproved. 

Whitens solutions to the first problem were to increase the size 
confirm the representativeness of thesaaple of subjects selected for 
learning hierarchy evaluations. Problem two was similar to Hively's 
objection and White suggested that this problem could be best solved 
throu^ introspection during the generation of a learning hierarchy 
candidate. However, no clues were given on how to detect this pro- 
blem during the evaluation phase of learning hierarchy research. 

Problems three and four are related. The use of only one ques- 
tion per element does not allow the estimation of the error in 
measuring the dependency among elements. Moreover, none of the existing 
dependency indexes reviewed by Capie aid Jones (1971) make the concept 
of error of. measurement explicit. Also, White (1974) charged that 
indexes su<:h as Gagne and Paradise's (1961) proportion of positive 
transfer as well as variants of it proposed by Walbeoser and Elsenberg 
(1972), Guttman's coefficient of reproducibility (used In Resnlck & 
Wang, 1969), and the four-fold contingency table and related phi corre- 
lation coefficient suggested by Capie and Jones (1971) share a common 
problem: each of these indexes can have values that indicate a hier- 
archical connection even when the skills are really independent. 
Certainly, the same charge could be directed at the Bart, et al., 
ordering- theoretic method. 

White and Clark (1973) introduced a statistical test of hierar- 
chical dependency which they purport treats the problem of error of 
measurement and also allows the inference of hierarchical dependencies 
among objectives for some population of interest. White and Clark 
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carefully discussed the power function for this significance test In 
their P^ydiomctlLka paper. Unfortunately, Vftiite failed to discuss 
power Issues in subsequent articles al»ed at less technically sophis- 
ticated audiences. And it is precisely these audiences which repeatedly 
fail to recognize relationships between sample size and the p<wer of 
significance tests in their research work (see Meehl, 1967). Conse- 
quently, educationally valid connections among elements in a hierarchy 
may, and probably will, be rejected if standard Neyman-Pearson 
hypothesis testing procedures are used in experiments with large aanples 
of subjects that Wiite also suggested. This point must be explained 
to prav ^al research workers. 

To solve the fifth problem, White advocated the administration 
of test items at key points during a learning progran designed to 
teach the subject matter in the hypothesized sequence under consideraticn. 
Then, White and Clark's significance test may be applied to determine 
whether dependencies exists among elen«nts in a posited hierarchy. 
Of course, thlf raises a fidelity issue: Does it aake a difference 
that the hierarchy under consideration may not ultimately be used to 
sequence a learning program? Perhaps this question merits investigation. 

The statement of White's sixth problem seems to be a reaction to 
blind, "dustbowl" empirlcisni In learning hierarchy research. White 
claimed that learning hierarchies modified on the basis of evaluative 
information should also be required to make logical. Intuitive sense. 
If this Is so, then what purpose does the evaluation serve and how 
are decisions to be made with an appropriate mix of logical and 
empirical informatin? On the other hand, unexamined learning hier- 
archies may have an intuitive appeal but may also seriously lack 
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ewpirical inport. As has already been asserted in this paper, the 
explication of decisions and decision-making strategies for the io- 
provement of learning hierarchies would be a noteworthy contribution 
to the literature. 

Concluding Remarks 

The need for an ea^irically defensible aeans of seqiancing 
instruction appears to have been the primary motivator for research 
into learning hierarchic Also, valid learning hierarchies could 
act as congealing forces in individualizing instructional systems 
by providing psychological roadmaps for diagnosing sttKients* pre- 
instructional skills and for monitoring students' progress throu^ 
Instructional systems. Pour methods for generating candidates for 
learning hierarchies were reviewed! introspection, fonaal analysis, 
observation, and statistical "fishing". Experimental transfer of 
training studies and statlscial studies have bean used to evaluate 
posited hiei -irchies. White suggented significant modifications 
of these standard evaluative m^^thods to improve the internal validity 
of research into learning hierarchies. 

Sliould We Contuiue. UueMAck Into Lzaming HUwdii<uf 

The study of learning hierarchies has all the trappings of what 
Kidin (19701 see also. Schwab, 1969) has called normal science: "Normal 
science means research firmly based upon one or more past scientific 
achievements, achievements that some particular scientific coimnunity 
acknowledges for a time as supplying the foundation for its further 
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practice iKuhn, 1970, p. 10].** There are three foci for noraal science: 
gathering facts to generate theory, predicting new facts froa theory, 
and reducing dissonance between the theory and the real world (Kuhn, 
1970, p. 25-27). Kuhn uses the tern paradiga to Indicate the parti- 
cular research tradition chosen to conduct normal science. The 
acquisition of a paradigm by an area of inquiry is a sign of the area's 
scientific maturity. Paradigms are also prescriptive since they 
govern the range of research problems acceptable to the scientific 
community. Moreover, they dictate the topics and modes of inquiry 
which students must master for acceptance and recognition in the 
scientific community. Studies in the sociology of science Indicate 
that the scope and structure of paradigms are controlled by visible 
and revered statesmen in an area of inquiry who, in turn, influence 
the behavior of younger, less prestigious researchers struggling 
below them. 

In terms of the socioloKT of psychology as a science, the impor- 
tance of Robert Gagne to the current interest in learning hierarchy 
research is not trivial. Gagne certainly fits the elder stateman's 
role in psychology. Would the study of learning hierarchies have 
been Important if Gagne had not chosen to become Involved? Of course 
an equally valid question is whether learning hierarchy research 
would have been discovered or received any creative impetus without 
his Involvement? More generally, are topics in instructional science 
researched because of the in^ortance of their patrons or are they 
studied for their intrinsic importance? Unraveling the answers to 
these questions may serve to demystify the study of learning hierarchies 
and lead to the statement of several critical questions for the future 
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of learning hierarchy research. 

Could we trim away our Interest in learning hierarchies and, 
thereby, achieve a more parsiiaonious instructional science? Could 
we conduct the teaching/learning enterprise without learning hier- 
archies? Would any instructional sequence be better than none 
at least as good as one suggested by reaource-cwwuning learning 
hierarchy research? Answers to these crucial questions, and others, 
oay contribute evidence tor deteraining the external validity of 
learning hierarchy studies. I would contend that these questions 
are Just as iagjortant as the present exclusive and pervasive interest 
in the internal validity of learning hierarchy experi»«tts which 
has been reviewed in this paper. But these are issues which oust 
be expanded upon in st^sequent papers. 
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